GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods

Motivation: Over the last decade, numerous methods have been developed for inference of regulatory networks from gene expression data. However, accurate and systematic evaluation of these methods is hampered by the difficulty of constructing adequate benchmarks and the lack of tools for a differentiated analysis of network predictions on such benchmarks. Results: Here, we describe a novel and comprehensive method for in silico benchmark generation and performance profiling of network inference methods available to the community as an open-source software called GeneNetWeaver (GNW). In addition to the generation of detailed dynamical models of gene regulatory networks to be used as benchmarks, GNW provides a network motif analysis that reveals systematic prediction errors, thereby indicating potential ways of improving inference methods. The accuracy of network inference methods is evaluated using standard metrics such as precision-recall and receiver operating characteristic curves. We show how GNW can be used to assess the performance and identify the strengths and weaknesses of six inference methods. Furthermore, we used GNW to provide the international Dialogue for Reverse Engineering Assessments and Methods (DREAM) competition with three network inference challenges (DREAM3, DREAM4 and DREAM5). Availability: GNW is available at http://gnw.sourceforge.net along with its Java source code, user manual and supporting data. Supplementary information: Supplementary data are available at Bioinformatics online. Contact: [email protected]

Evolutionary reverse engineering of gene networks

The expression of genes is controlled by regulatory networks, which perform fundamental information processing and control mechanisms in a cell. Unraveling and modelling these networks will be indispensable to gain a systems-level understanding of biological organisms and genetically related diseases. In this thesis, we present an evolutionary reverse engineering method, which allows to simultaneously infer both the wirings and nonlinear dynamical models of gene regulatory networks from gene expression data. The proposed method reconstructs gene networks by mimicking the natural evolutionary process that constructed them. This is achieved by modelling both the way in which gene networks are encoded in the biological genome, and the different types of mutations and recombinations that drive their evolution, using an artificial genome called Analog Genetic Encoding (AGE). Since AGE mimics the evolutionary forces and constraints that shape biological gene networks, the reconstruction is naturally guided towards biologically plausible solutions. Consequently, the search space is explored more efficiently, and the networks are recovered more reliably, than with alternative methods. We have confirmed the state-of-the-art performance of AGE both in vivo (on real gene networks) and in silico (on simulated networks). In particular, AGE achieved winning performance in the in vivo gene network inference inference challenge of the 2nd DREAM (Dialogue on Reverse Engineering Assessment and Methods) conference, which consisted in predicting the structure of a synthetic-biology gene network in Saccharomyces cerevisiae from time-series data. In vivo performance assessment of network-inference methods is problematic because it is in general not possible to systematically validate predictions, except for few well-characterized gene networks. Consequently, in silico benchmarks are essential to understand the performance of network-inference methods. We have developed tools to generate biologically plausible in silico gene networks, which allow realistic performance assessment of network-inference methods. In contrast to previous in silico benchmarks, we generate network structures by extracting modules from known gene networks of model organisms, instead of using random graphs. Furthermore, we simulate network dynamics using more realistic kinetic models, which include both mRNA and proteins. We have implemented this framework in an open-source Java tool called GeneNetWeaver (GNW). Using GNW we have generated benchmarks for community-wide challenges of the 3rd and 4th DREAM conference (the DREAM in silico network challenges). Here, we assess the performance of 29 network-inference methods, which have been applied independently by participating teams of the DREAM3 challenge. Performance profiling on individual network motifs reveals that current inference methods are affected, to various degrees, by three types of systematic prediction errors. We find that these errors are induced by inaccurate prior assumptions of prevalent gene-network models. The evolutionary reverse engineering approach, which would have ranked 3rd in this challenge, can be used with a wide range of nonlinear models. It could thus provide the necessary framework for the development of models that better approximate different types of gene regulation, thereby enabling ever more accurate reconstruction of gene networks

Wisdom of crowds for robust gene network inference

Reconstructing gene regulatory networks from high-throughput data is a long-standing challenge. Through the Dialogue on Reverse Engineering Assessment and Methods (DREAM) project, we performed a comprehensive blind assessment of over 30 network inference methods on Escherichia coli, Staphylococcus aureus, Saccharomyces cerevisiae and in silico microarray data. We characterize the performance, data requirements and inherent biases of different inference approaches, and we provide guidelines for algorithm application and development. We observed that no single inference method performs optimally across all data sets. In contrast, integration of predictions from multiple inference methods shows robust and high performance across diverse data sets. We thereby constructed high-confidence networks for E. coli and S. aureus, each comprising ~1,700 transcriptional interactions at a precision of ~50%. We experimentally tested 53 previously unobserved regulatory interactions in E. coli, of which 23 (43%) were supported. Our results establish community-based methods as a powerful and robust tool for the inference of transcriptional gene regulatory networks.National Institutes of Health (U.S.)National Centers for Biomedical Computing (U.S.) (Roadmap Initiative (U54CA121852))Howard Hughes Medical InstituteNational Institutes of Health (U.S.) (Director's Pioneer Award DPI OD003644)Swiss National Science Foundation (Fellowship

Tissue-specific regulatory circuits reveal variable modular perturbations across complex diseases

Mapping perturbed molecular circuits that underlie complex diseases remains a great challenge. We developed a comprehensive resource of 394 cell type– and tissue-specific gene regulatory networks for human, each specifying the genome-wide connectivity among transcription factors, enhancers, promoters and genes. Integration with 37 genome-wide association studies (GWASs) showed that disease-associated genetic variants—including variants that do not reach genome-wide significance—often perturb regulatory modules that are highly specific to disease-relevant cell types or tissues. Our resource opens the door to systematic analysis of regulatory programs across hundreds of human cell types and tissue

Combining Multiple Results of a Reverse Engineering Algorithm: Application to the DREAM Five Gene Network Challenge

The output of reverse engineering methods for biological networks is often not a single network prediction, but an ensemble of networks that are consistent with the experimentally measured data. In this paper, we consider the problem of combining the information contained within such an ensemble in order to (1) make more accurate network predictions and (2) estimate the reliability of these predictions. We review existing methods, discuss their limitations, and point out possible research directions towards more advanced methods for this purpose. The potential of considering ensembles of networks, rather than individual inferred networks, is demonstrated by showing how an ensemble voting method achieved winning performance on the Five Gene Network Challenge of the second DREAM conference (Dialogue on Reverse Engineering Assessment and Methods 2007, New York, NY)

Replaying the Evolutionary Tape: Biomimetic Reverse Engineering of Gene Networks

In this paper, we suggest a new approach for reverse engineering gene regulatory networks, which consists of using a reconstruction process that is similar to the evolutionary process that created these networks. The aim is to integrate prior knowledge into the reverse engineering procedure, thus biasing the search towards biologically plausible solutions. To this end, we propose an evolutionary method that abstracts and mimics the natural evolution of gene regulatory networks. Our method can be used with a wide range of nonlinear dynamical models. This allows us to explore novel model types such as the log-sigmoid model introduced here. To allow direct comparison with other methods, we use a benchmark dataset from an in vivo synthetic-biology gene network, which has been published as a reverse engineering challenge for the second DREAM conference

Generating Realistic In Silico Gene Networks for Performance Assessment of Reverse Engineering Methods

Reverse engineering methods are typically first tested on simulated data from in silico networks, for systematic and efficient performance assessment, before an application to real biological networks. In this paper we present a method for generating biologically plausible in silico networks, which allow realistic performance assessment of network inference algorithms. Instead of using random graph models, which are known to only partly capture the structural properties of biological networks, we generate network structures by extracting modules from known biological interaction networks. Using the yeast transcriptional regulatory network as a test case, we show that extracted modules have a biologically plausible connectivity because they preserve functional and structural properties of the original network. Our method was selected to generate the "gold standard" networks for the gene network reverse engineering challenge of the third DREAM conference (Dialogue on Reverse Engineering Assessment and Methods, Cambridge, MA, 2008)